A relation between Shannon-Weaver entropy and "theoretical dimension" for classes of smooth functions

Abstract: "Suppose that an infinite sequence is produced by independent trials of a random variable with a fixed distribution. The Shannon-Weaver entropy of the sequence determines the minimum bit rate needed to transmit the values of the sequence. We show that if the source distribution is highly concentrated, as is commonly observed in practice, then its entropy is equal to the logarithm of the theoretical dimension of the sequence. We conclude that the best-basis algorithm, which minimizes this theoretical dimension over a library of transformations, both chooses the transformation that yields best compression and also gives an estimate of the compression rate.